{
 "cells": [
  {
   "cell_type": "markdown",
   "id": "0",
   "metadata": {},
   "source": [
    "[![image](https://jupyterlite.rtfd.io/en/latest/_static/badge.svg)](https://demo.leafmap.org/lab/index.html?path=notebooks/87_actinia.ipynb)\n",
    "[![image](https://colab.research.google.com/assets/colab-badge.svg)](https://colab.research.google.com/github/opengeos/leafmap/blob/master/docs/notebooks/87_actinia.ipynb)\n",
    "[![image](https://mybinder.org/badge_logo.svg)](https://mybinder.org/v2/gh/opengeos/leafmap/HEAD)\n",
    "\n",
    "**Cloud-based geoprocessing with Actinia**"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "1",
   "metadata": {},
   "outputs": [],
   "source": [
    "# %pip install -U leafmap"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "2",
   "metadata": {},
   "source": [
    "The cloud based geoprocessing platform [actinia](https://github.com/actinia-org) is able to ingest and analyse large volumes of geodata in the cloud.\n",
    "\n",
    "For the following actinia example we use the [actinia-python-client](https://actinia-org.github.io/actinia-python-client/) ([source code](https://github.com/actinia-org/actinia-python-client)) to establish the connection to an actinia instance. \n",
    "First install the actinia-python-client (for latest version, see [actinia-python-client releases](https://github.com/actinia-org/actinia-python-client/releases))."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "3",
   "metadata": {},
   "outputs": [],
   "source": [
    "import os\n",
    "import leafmap"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "4",
   "metadata": {},
   "outputs": [],
   "source": [
    "%pip install actinia_python_client"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "5",
   "metadata": {},
   "source": [
    "The results of [actinia](https://actinia.mundialis.de/) ephemeral processing are available via object storage as GeoTIFF/COG or GeoPackage files."
   ]
  },
  {
   "cell_type": "markdown",
   "id": "6",
   "metadata": {},
   "source": [
    "Add a helper function for \"pretty printing\" of actinia results:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "7",
   "metadata": {},
   "outputs": [],
   "source": [
    "from json import dumps as json_dumps\n",
    "\n",
    "\n",
    "def print_dict(input_dict, text=None):\n",
    "    if text:\n",
    "        print(text)\n",
    "    if \"region\" in input_dict:\n",
    "        input_dict[\"region\"] = input_dict[\"region\"].__dict__\n",
    "    print(json_dumps(input_dict, sort_keys=True, indent=4))\n",
    "\n",
    "\n",
    "def print_dict_keys(input_dict, text=None):\n",
    "    if text:\n",
    "        print(text)\n",
    "    print(\", \".join(input_dict.keys()))"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "8",
   "metadata": {},
   "source": [
    "Connect to the default actinia server which is defined in the actinia-python-client, currently https://actinia.mundialis.de."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "9",
   "metadata": {},
   "outputs": [],
   "source": [
    "# connect to the actinia server\n",
    "from actinia import Actinia\n",
    "\n",
    "# connect to default actinia server (https://actinia.mundialis.de)\n",
    "actinia_mundialis = Actinia()\n",
    "\n",
    "# retrieve metadata about actinia server and related software versions\n",
    "version = actinia_mundialis.get_version()\n",
    "print_dict(version, \"Version is:\")"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "10",
   "metadata": {},
   "source": [
    "Set the authentication settings of the actinia demo user to gain access to the actinia server functionality."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "11",
   "metadata": {},
   "outputs": [],
   "source": [
    "actinia_user = \"demouser\"\n",
    "actinia_password = \"gu3st!pa55w0rd\"\n",
    "\n",
    "# we use the default actinia server\n",
    "actinia_mundialis.set_authentication(actinia_user, actinia_password)\n",
    "print(\"Connected to actinia server.\")"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "12",
   "metadata": {},
   "source": [
    "Obtain the list of locations and retrieve the metadata of a selected location."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "13",
   "metadata": {},
   "outputs": [],
   "source": [
    "# obtain the list of projects (called \"locations\") which are accessible to current user\n",
    "locations = actinia_mundialis.get_locations()\n",
    "print_dict_keys(locations, \"Locations: \")"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "14",
   "metadata": {},
   "source": [
    "Retrieve the metadata of a selected location (this shows the respective projection information, spatial extent, resolution, etc.) to get an idea how the output looks like."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "15",
   "metadata": {},
   "outputs": [],
   "source": [
    "print_dict(actinia_mundialis.locations[\"nc_spm_08\"].get_info(), \"Location info:\")"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "16",
   "metadata": {},
   "source": [
    "At this point the connection to the selected actinia server is properly established."
   ]
  },
  {
   "cell_type": "markdown",
   "id": "17",
   "metadata": {},
   "source": [
    "**Reading the online data resource into the actinia server**"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "18",
   "metadata": {},
   "source": [
    "Next we demonstrate the data processing of a raster map available online in actinia, here a sample DEM GeoTIFF file."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "19",
   "metadata": {},
   "outputs": [],
   "source": [
    "# define raster elevation map name\n",
    "raster_layer_name = \"srtm90\"\n",
    "\n",
    "# cache file locally\n",
    "out_dir = os.getcwd()\n",
    "dem_file = os.path.join(out_dir, f\"{raster_layer_name}.tif\")\n",
    "\n",
    "# dem_url = (\n",
    "#    \"https://drive.google.com/file/d/1vRkAWQYsLWCi6vcTMk8vLxoXMFbdMFn8/view?usp=sharing\"\n",
    "# )\n",
    "dem_url = f\"https://github.com/giswqs/data/raw/main/raster/{raster_layer_name}.tif\"\n",
    "\n",
    "# leafmap.download_file(dem_url, dem_file, unzip=False, overwrite=True)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "20",
   "metadata": {},
   "source": [
    "Prepare actinia location and mapset, i.e. generate a subproject for data processing."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "21",
   "metadata": {},
   "outputs": [],
   "source": [
    "# request list of all locations\n",
    "locations = actinia_mundialis.get_locations()\n",
    "print([loc for loc in locations])"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "22",
   "metadata": {},
   "outputs": [],
   "source": [
    "# remove leftover location from previous run\n",
    "# actinia_mundialis.locations[\"latlong_wgs84\"].delete()\n",
    "#\n",
    "# remove leftover mapset from previous run\n",
    "locations[\"latlong_wgs84\"].delete_mapset(\"elevation\")"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "23",
   "metadata": {},
   "outputs": [],
   "source": [
    "# Create a new location for the data processing in actinia\n",
    "new_location = actinia_mundialis.create_location(\"latlong_wgs84\", 4326)\n",
    "print(new_location.name)\n",
    "print(new_location.region)\n",
    "print([loc for loc in actinia_mundialis.locations])\n",
    "\n",
    "# request list of mapsets in selected location\n",
    "mapsets = actinia_mundialis.locations[\"latlong_wgs84\"].get_mapsets()\n",
    "print_dict_keys(mapsets, \"Mapsets in latlong_wgs84:\")"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "24",
   "metadata": {},
   "outputs": [],
   "source": [
    "# Create a new mapset for the data processing in actinia\n",
    "mapset_name = \"elevation\"\n",
    "locations[\"latlong_wgs84\"].create_mapset(mapset_name)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "25",
   "metadata": {},
   "outputs": [],
   "source": [
    "## Optional: Upload the sample DEM data set to actinia (indeed not needed since we use `vsicurl/` below\n",
    "##           to directly retrieve the online dataset).\n",
    "# locations[\"latlong_wgs84\"].mapsets[mapset_name].upload_raster(raster_layer_name, dem_file)\n",
    "# print_dict_keys(locations[\"latlong_wgs84\"].mapsets[mapset_name].raster_layers, \"Raster maps in new mapset:\")"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "26",
   "metadata": {},
   "source": [
    "#### Ephemeral Processing with actinia\n",
    "\n",
    "**Ephemeral processing** is used to keep computed results, including user-generated data and temporary data, only for a limited period of time (e.g. 24 hours by default in the actinia demo server). This reduces cloud storage costs.\n",
    "\n",
    "In contrast, **persistent processing** refers to the persistent retention of data without a scheduled deletion time, even in the event of a power outage, resulting in corresponding storage costs. In the geo/EO context, persistent storage is used to provide, for example, basic cartography, i.e. elevation models, road networks, building footprints, etc."
   ]
  },
  {
   "cell_type": "markdown",
   "id": "27",
   "metadata": {},
   "source": [
    "**Hillshading example**\n",
    "\n",
    "Here an example for an ephemeral processing job: We download and import the remotely available GeoTIFF file. Then we use [r.relief](https://grass.osgeo.org/grass-stable/manuals/r.relief.html) to generate a hillshading map and pre-define the resolution to 10 m. The computational region is set to the input elevation map. The resulting `hillshade.tif` raster map is then provided as a resource for download and visualization."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "28",
   "metadata": {},
   "outputs": [],
   "source": [
    "pc = {\n",
    "    \"list\": [\n",
    "        {\n",
    "            \"id\": \"importer_0\",\n",
    "            \"comment\": \"Import of remote data source (here: COG)\",\n",
    "            \"module\": \"r.import\",\n",
    "            \"inputs\": [\n",
    "                {\"param\": \"input\", \"value\": f\"/vsicurl/{dem_url}\"},\n",
    "                {\"param\": \"memory\", \"value\": \"2000\"},\n",
    "                {\"param\": \"extent\", \"value\": \"input\"},\n",
    "            ],\n",
    "            \"outputs\": [{\"param\": \"output\", \"value\": f\"{raster_layer_name}\"}],\n",
    "        },\n",
    "        {\n",
    "            \"id\": \"r.info_1\",\n",
    "            \"comment\": \"Print metadata of imported raster map\",\n",
    "            \"module\": \"r.info\",\n",
    "            \"inputs\": [{\"param\": \"map\", \"value\": f\"{raster_layer_name}\"}],\n",
    "        },\n",
    "        {\n",
    "            \"id\": \"computational_region_2\",\n",
    "            \"comment\": \"Set computational region to imported map, and print settings\",\n",
    "            \"module\": \"g.region\",\n",
    "            \"inputs\": [{\"param\": \"raster\", \"value\": f\"{raster_layer_name}\"}],\n",
    "            \"stdout\": {\"id\": \"region\", \"format\": \"kv\", \"delimiter\": \"=\"},\n",
    "            \"flags\": \"g\",\n",
    "        },\n",
    "        {\n",
    "            \"id\": \"create_hillshading_3\",\n",
    "            \"comment\": \"Compute hillshading map\",\n",
    "            \"module\": \"r.relief\",\n",
    "            \"inputs\": [{\"param\": \"input\", \"value\": f\"{raster_layer_name}\"}],\n",
    "            \"outputs\": [{\"param\": \"output\", \"value\": \"hillshade\"}],\n",
    "        },\n",
    "        {\n",
    "            \"id\": \"exporter_4\",\n",
    "            \"comment\": \"Export hillshading map to COG file\",\n",
    "            \"module\": \"exporter\",\n",
    "            \"outputs\": [\n",
    "                {\n",
    "                    \"export\": {\"type\": \"raster\", \"format\": \"COG\"},\n",
    "                    \"param\": \"map\",\n",
    "                    \"value\": \"hillshade\",\n",
    "                }\n",
    "            ],\n",
    "        },\n",
    "    ],\n",
    "    \"version\": \"1\",\n",
    "}\n",
    "\n",
    "\n",
    "print(pc)\n",
    "job = actinia_mundialis.locations[\"latlong_wgs84\"].create_processing_export_job(\n",
    "    pc, \"hillshading\"\n",
    ")\n",
    "job.poll_until_finished()\n",
    "\n",
    "print(job.status)\n",
    "print(job.message)\n",
    "exported_raster = job.urls[\"resources\"][0]\n",
    "print(exported_raster)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "29",
   "metadata": {},
   "source": [
    "It will take a moment, then the communication by actinia is shown: \"Status of hillshading job is accepted: Resource accepted\" continued by further communication messages."
   ]
  },
  {
   "cell_type": "markdown",
   "id": "30",
   "metadata": {},
   "source": [
    "In case an error occurs, check the process log (use [x] with x being the step number in the process chain). Examples:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "31",
   "metadata": {},
   "outputs": [],
   "source": [
    "# check step 0 (r.import)\n",
    "print_dict(job.process_log[0])"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "32",
   "metadata": {},
   "outputs": [],
   "source": [
    "# check step 2 (g.region)\n",
    "print_dict(job.process_log[2])"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "33",
   "metadata": {},
   "outputs": [],
   "source": [
    "# check step 3 (r.relief)\n",
    "print_dict(job.process_log[3])"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "34",
   "metadata": {},
   "source": [
    "Inject `user:password@server` into `exported_raster` URL (i.e., the actinia resource)."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "35",
   "metadata": {},
   "outputs": [],
   "source": [
    "url = exported_raster.replace(\"//\", f\"//{actinia_user}:{actinia_password}@\")\n",
    "print(url)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "36",
   "metadata": {},
   "source": [
    "Visualize the `hillshade` map in leafmap (colorbar inspired by [this notebook](https://leafmap.org/notebooks/07_colorbar/)):"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "37",
   "metadata": {},
   "outputs": [],
   "source": [
    "m = leafmap.Map()\n",
    "m.add_basemap(\"OpenTopoMap\")\n",
    "m.add_colormap(\n",
    "    cmap=\"terrain\",\n",
    "    label=\"Elevation\",\n",
    "    width=3,\n",
    "    height=0.3,\n",
    "    orientation=\"horizontal\",\n",
    "    vmin=0,\n",
    "    vmax=4000,\n",
    ")\n",
    "m.add_cog_layer(\n",
    "    url,\n",
    "    name=\"SRTM90 hillshaded map\",\n",
    "    attribution='<a href=\"https://e4ftl01.cr.usgs.gov/MEASURES/\">https://e4ftl01.cr.usgs.gov/MEASURES/</a>',\n",
    ")\n",
    "# show map\n",
    "m"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "38",
   "metadata": {},
   "source": [
    "Find further leafmap (styling) tools in the upper-right toolbox of leafmap."
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 5
}
